Three-dimensional image reconstruction using multi-layer data acquisition

ABSTRACT

A camera, including: two imaging systems each comprising a different optical path corresponding to a different viewing angle of an object; one or more illumination sources; a mask disposed with multiple pairs of apertures, wherein each aperture of each aperture pair corresponds to a different one of the imaging systems; at least one detector configured to acquire multiple image pairs of the object from the two imaging systems via the multiple pairs of apertures; and a processor configured to produce from the multiple acquired image pairs a multi-layer three dimensional reconstruction of the object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Phase of PCT Patent Application No.PCT/IL2017/051374 having International filing date of Dec. 21, 2017,which claims the benefit of priority of U.S. Provisional PatentApplication No. 62/437,881 filed Dec. 22, 2016. The contents of theabove applications are all incorporated by reference as if fully setforth herein in their entirety.

BACKGROUND

The invention relates to the field of three-dimensional (3D) imaging.

Conventional cameras transform a three-dimensional view of an objectinto a two dimensional image. Typically, the depth dimension,corresponding to the distance between the focal plane of the capturedimage and the camera, is lost. To include a depth characteristic, someoptical systems use two cameras to capture a pair of stereo images ofthe object, much the way our eyes work. Each image of the pair isacquired from a slightly different viewing angle, and the discrepancybetween the two images is used to measure depth.

The foregoing examples of the related art and limitations relatedtherewith are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the figures.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems, tools and methods which aremeant to be exemplary and illustrative, not limiting in scope.

One embodiment provides a camera, comprising: two imaging systems eachcomprising a different optical path corresponding to a different viewingangle of an object; one or more illumination sources; a mask comprisingmultiple pairs of apertures, wherein each aperture of each aperture paircorresponds to a different one of the imaging systems; at least onedetector configured to acquire multiple image pairs of the object fromthe two imaging systems via the multiple pairs of apertures; and aprocessor configured to produce, from the multiple acquired image pairs,a multi-layer three dimensional reconstruction of the object.

In some embodiments, said one or more illumination sources are multipleillumination sources, and wherein each pair of apertures is dedicated toa different one of the multiple illumination sources.

In some embodiments, said multiple illumination sources comprise a redLED, a green LED, and a blue LED.

In some embodiments, each of the multiple different LEDs has afull-width-at-half (FWHM) bandwidth of 10 nm±10%.

In some embodiments, the at least one detector comprises a mono sensorhaving a dedicated region for each aperture, and configured tosimultaneously capture multiple images comprising an image for each ofthe multiple illumination sources and for each of the imaging systems.

In some embodiments, the at least one detector comprises two imagingsensors that are each dedicated to one of the imaging systems and areconfigured to simultaneously detect an image pair for each of theillumination sources.

In some embodiments, the at least one detector comprises an imagingsensor having multiple sensor regions, each sensor region dedicated to adifferent one of the multiple apertures, and wherein the multiple sensorregions are configured to simultaneously capture multiple imagescomprising an image for each of the multiple illumination sources andfor each of the imaging systems.

In some embodiments, at least one of the multiple different illuminationsources is disposed with a polarization component, and wherein at leastone of the apertures dedicated to the illumination source disposed withthe polarization component, is disposed with a complementarypolarization component.

In some embodiments, the at least one of the apertures disposed with thepolarization component is dedicated to a blue light source.

In some embodiments, each pair of apertures is disposed with a colorfilter corresponding to its dedicated illumination source.

In some embodiments, the camera further comprises at least one back lensconfigured to focus light transmitted via the multiple pairs ofapertures of the mask to the at least one detector.

In some embodiments, at least one image pair of the acquired image pairscorresponds to a specular reflection off the object, and wherein theprocessor is further configured to use the at least one image pair tomeasure a depth characteristic of the object.

In some embodiments, the processor is further configured to apply ashift to the pixels of the acquired image as a function of the depthcharacteristic.

In some embodiments, at least one of the acquired images corresponds toa diffuse reflection off the object, and wherein the processor isfurther configured to use the at least one acquired image to measure ahemoglobin level of the object.

In some embodiments, at least one of the acquired images corresponds toa diffuse reflection off the object, and wherein the processor isfurther configured to use the at least acquired one image to measure amelanin level of the object.

In some embodiments, the processor is configured to use at least one ofthe acquired images to measure a color characteristic of the object.

In some embodiments, the processor is configured to synchronize theillumination source with a shutter of the camera.

In some embodiments, the camera further comprises a user interfaceconfigured to display the multi-layer three dimensional reconstructionof the object.

Another embodiment provides an optical mask, comprising: multipleaperture pairs, wherein each aperture pair is dedicated to a differentillumination wavelength, wherein each aperture of each aperture pair isconfigured to transmit a light pulse from a different one of two opticalpaths to at least one imaging sensor.

In some embodiments, at least one aperture is disposed with apolarization component.

In some embodiments, each aperture pair has a different size.

In some embodiments, each aperture pair has a different shape.

In some embodiments, one aperture pair shape is round and wherein oneaperture pair shape is square, and wherein one aperture pair shape isrectangular.

In some embodiments, each aperture pair is disposed with a color filtercorresponding to its dedicated illumination wavelength.

Another embodiment provides a method comprising: controlling anillumination cycle comprising a blue illumination pulse by a blue lightsource, a green illumination pulse by a green light source, and a redillumination pulse by a red light source; synchronizing a camera shutterwith each illumination pulse of the illumination cycle; acquiring a pairof images of an object during each illumination pulse; calculating adepth characteristic of the object using one of the pairs of imagesacquired from a specular reflection of one of illumination pulses;applying a shift to the pixels of the acquired image as a function ofthe depth characteristic; measuring a sub-surface quality of the objectusing any of images acquired from a diffuse reflection of any of theillumination pulses; determining a color characteristic of the objectfrom any of the acquired images; registering the pairs of imagesreceived over multiple illumination cycles; and combining the registeredpairs of images to create a multi-layer three-dimensional reconstructionof the object.

In some embodiments, the specular reflection corresponds to the blueillumination pulse.

In some embodiments, the diffuse reflection corresponds to any of thered and green illumination pulses.

In some embodiments, the pairs of images are acquired from two regionsof a mono sensor.

In some embodiments, one of the images is acquired from polarized light.

In some embodiments, the polarized light corresponds to the blueillumination pulse.

In some embodiments, the polarized light corresponds to any of the greenand red illumination pulses.

In some embodiments, the object is skin and the sub-surface quality is ahemoglobin level of the skin.

In some embodiments, the object is skin and the sub-surface quality is amelanin level of the skin.

In some embodiments, the color characteristic is determined from thepairs of images acquired during the blue illumination pulse.

In some embodiments, the color characteristic is determined from oneimage from each pair of images acquired during each of the blue, green,and red illumination pulses.

In some embodiments, the method further comprises applying to theregistered pairs of images any of: an edge enhancement technique, anequalization technique, and an image correlation technique comprising:a) calculating a variance map for each color, b) scaling the variancemaps to the same dynamic range, and c) finding the relative positions ofdetected features to maximize correlation.

In some embodiments, the method further comprises displaying thethree-dimensional reconstruction.

Another embodiment provides a method comprising: simultaneouslyilluminating with a blue illumination pulse by a blue light source, agreen illumination pulse by a green light source, and a red illuminationpulse by a red light source; synchronizing a camera shutter with thesimultaneous illumination pulses; simultaneously acquiring a pair ofimages for each illumination pulse; calculating a depth characteristicof the object using one of the pairs of images acquired from a specularreflection of one of illumination pulses; applying a shift to the pixelsof the acquired image as a function of the depth characteristic;measuring a sub-surface quality of the object using any of imagesacquired from a diffuse reflection of any of the illumination pulses;determining a color characteristic of the object from any of theacquired images; registering the pairs of images received over multipleillumination cycles; and combining the registered pairs of images tocreate a multi-layer three-dimensional reconstruction of the object.

In some embodiments, the specular reflection corresponds to the blueillumination pulse.

In some embodiments, the diffuse reflection corresponds to any of thered and green illumination pulses.

In some embodiments, the multiple pairs of images are acquired bymultiple dedicated regions of a mono sensor.

In some embodiments, the multiple pairs of images are acquired bymultiple dedicated sensors.

In some embodiments, one of the images is acquired from polarized light.

In some embodiments, the polarized light corresponds to the blueillumination pulse.

In some embodiments, the polarized light corresponds to any of the greenand red illumination pulses.

In some embodiments, the object is skin and the sub-surface quality is ahemoglobin level of the skin.

In some embodiments, the object is skin and the sub-surface quality is amelanin level of the skin.

In some embodiments, the color characteristic is determined from thepairs of images acquired during the blue illumination pulse.

In some embodiments, the color characteristic is determined from oneimage from each pair of images acquired during each of the blue, green,and red illumination pulses.

The method of claim 38, further comprising applying to the registeredpairs of images any of: an edge enhancement technique, an equalizationtechnique, and an image correlation technique comprising: a) calculatinga variance map for each color, b) scaling the variance maps to the samedynamic range, and c) finding the relative positions of detectedfeatures to maximize correlation.

In some embodiments, the method further comprises displaying thethree-dimensional reconstruction.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thefigures and by study of the following detailed description.

BRIEF DESCRIPTION OF THE FIGURES

Exemplary embodiments are illustrated in referenced figures. Dimensionsof components and features shown in the figures are generally chosen forconvenience and clarity of presentation and are not necessarily shown toscale. The figures are listed below.

FIGS. 1A-1B, taken together, an optical imaging system in accordancewith an embodiment;

FIGS. 2A-2B illustrate an optical imaging system in accordance withanother embodiment;

FIG. 3 illustrates reflective properties of human skin;

FIG. 4A shows the spectral response of various optical Bayer filters;

FIG. 4B shows the illumination bandwidths of various color lightemitting diodes (LEDs);

FIGS. 5A-5C show multiple images of human skin captured using varyingillumination and sensors;

FIG. 6 shows visibility as a function of distance from a camera using amono sensor and color sensor;

FIGS. 7A-7C show images of human skin captured using a mono sensor underblue illumination with no polarization, parallel polarization, andcross-polarization;

FIGS. 8A-8B show images of human skin captured using parallel polarizersand crossed polarizers, respectively;

FIGS. 8C and 8D show images of human skin captured under green and redillumination, respectively, with cross polarizers;

FIGS. 9A-9D, taken together, illustrate a system for multi-layer imageacquisition for a 3D reconstruction of an object in accordance with anembodiment;

FIGS. 9E-9F show a configuration of an optical apparatus configured withthe system of FIG. 9A, in accordance with an embodiment;

FIG. 9G shows another configuration of an optical apparatus configuredwith the system of FIG. 9A, in accordance with an embodiment;

FIGS. 10A-10D show three cycles for multi-layer image acquisition usingthe mask of FIG. 9C;

FIGS. 11A-11C show images of the same skin surface captured under blue,green, and red illumination, respectively;

FIG. 12 shows an exemplary multi-layer 3D reconstruction using thesystem and method described herein; and

FIGS. 13A-B taken together, show a flowchart of a method for creating amulti-layer 3D reconstruction of an object, in accordance with anembodiment.

DETAILED DESCRIPTION

A system and method are disclosed herein for 3D image reconstruction.Multiple images may be captured in stereo under varying polarization andcolor illuminations to obtain different perspective views andcharacteristics of a three dimensional (3D) object. The imaging systemmay be used to image a tissue such as skin. Since skin is slightlytranslucent, illuminating and imaging with varying polarizations and/orwavelengths may allow acquiring images via diffuse reflection, to imagecharacteristics beneath the skin's surface, such as hemoglobin andmelanin concentrations. Additionally, imaging in stereo using surface,primarily specular, reflection may provide both color and depthcharacteristics at the skin's surface indicating scars, lines, and/orother surface details. The multiple images acquired in stereo undervarying polarization and color illuminations may be combined to create amulti-layered 3D reconstruction of the skin's surface.

Reference is now made to FIGS. 1A-1B which, taken together, show anoptical imaging system in accordance with an embodiment.

An imaging system 100, such as a camera, is provided to capture multiplestereo images of a 3D object 110, such as skin. Camera 100 is providedwith a front (objective) lens 102 for collecting light reflected offobject 110. The collected light is transmitted through one or moreapertures of a mask 104 to a pair of back lenses 106 a and 106 b, whichfocus the collected light onto one or more sensors 108 a, 108 b, such asmay comprise any suitable imaging sensor, for example a charged coupleddevice (CCD) or a complementary metal-oxide semiconductor (CMOS). Thecaptured images may be transmitted to a processor 112 which uses thecaptured images to reconstruct a 3D image of object 110.

Referring to FIG. 1B, an exemplary view of sensor 108 is shown,according to an embodiment. Sensors 108 a, 108 b may be implemented astwo distinct regions, 108 a and 108 b of sensor 108. Alternatively, twoindividual sensors 108 a and 108 b may be provided for each of theimaging systems.

Camera 100 may constitute two imaging systems, each system comprisingone of lenses 106 a and 106 b and one of sensors 108 a and 108 b, andsharing common objective 102. The two imaging systems may allow imagingalong separate optical paths, each corresponding to a different one ofviewing angles θ₁, θ₂ off of object 110, thereby allowing simultaneousstereo image acquisition. Each of lenses 106 a and 106 b and sensors 108a and 108 b may be dedicated to one of the optical paths. This designmay allow the overall length of the imaging systems to be constrainedwithin a predefined size limitation, such as for implementing within ahand-held device. Alternatively, a separate objective (not shown) may beprovided for each optical path.

The imaging systems may be telecentric in the sample space of object 110such as by positioning mask 104 in the rear focal plane of objective102, allowing to decouple the defocusing and magnification of object110. Optionally, back lenses 106 a and 106 b may operate undertelecentric conditions such as by positioning mask 104 in the frontfocal plane of back lenses 106 a and 106 b. Optionally, the distancebetween mask 104 to back lenses 106 a and 106 b may be less than orgreater than the focal length of back lenses 106 a and 106 b to controlthe imaging properties of the system, for example the distortion.Optionally, the distance between mask 104 to back lenses 106 a and 106 bmay be set to be less than the focal length of back lenses 106 a and 106b, such that the chief ray expands at a suitable angle when incidentupon the corners of sensors 108 a and 108 b, to provide an optimumcondition for light level uniformity, as is commonly found in manymodern image sensors.

The telecentric imaging described thus may allow for uniform scaledimaging, and thus, regions of object 110 positioned either above orbelow the best-focus plane may be imaged at the same size-scale asregions positioned at the optimum focus. This property may be usefulwhen combining the multiple different captured images by processor 112for performing the 3D reconstruction of object 110.

Mask 104 may have two or more apertures for transmitting the collectedlight, each aperture corresponding to a different one of the opticalpaths. In one implementation, mask 104 includes a pair of round holes toproduce the desired F-number (F #) at object 110, such as illustrated inFIG. 1A, where F # is understood to control: the amount of lightcollected by imaging system 100; the lateral resolution of imagingsystem 100; and the depth-of-focus of imaging system 100. Optionally,mask 104 provides a high F # to produce a large depth-of-focus bycausing the image of a sensor pixel to expand slowly above and belowbest focus on object 110, as is known in the art.

System 100 may be designed to image object 110 positioned at or near thefront focal plane of the front lens such that sensors 108 a and 108 band lenses 106 a and 106 b are operating at infinite conjugates. Thus,light reflected off object 110 at angle θ₁ may be collimated viaobjective 102 through one aperture of mask 104 and focused via lens 106a onto sensor 108 a, and light reflected off sample 110 at angle θ₂ maybe collimated via objective 102 through a second aperture of mask 104and focused via lens 106 b onto sensor 108 b. In this manner, differentpoints on object 110 imaged at different angles θ₁, θ₂ may be mappedonto different regions of the mask plane and different regions of thesensor plane, comprising a different imaging system for each viewingangle. Similarly, light rays reflecting off a single point of object 110at different angles θ₁, θ₂ may be parallel when they arrive at mask 104,and transmitted, respectively through the different apertures via backlenses 106 a and 106 b to sensors 108 a and 108 b. In this manner, thetwo imaging systems together allow the simultaneous stereo imaging frommultiple different angular views of object 110. Optionally, each viewingangle may be imaged sequentially at sensor 108.

The apertures on mask 104 may be positioned symmetrically opposite aboutthe viewing axis of camera 100, allowing two slightly different views ofthe 3D surface to be obtained. The disparity Δ between the two differentcaptured views may be computed and used to determine a depth attributeof the imaged 3D surface. The disparity may be computed as thedifferences between the lateral (X, Y) positions of one or moreidentified features in the two images. A 3D map of the imaged object maybe formed by computing the disparity between each identified feature inthe two captured views. The disparity may be computed using any suitablealgorithm such as are known in the art of stereoscopy. The depth can becalculated using the following equation:Z=F ₁ +ΔF ₁ ²/(F ₂ b)   (1)

Where F₁ is the front lens (objective) focal length, F₂ is the back lensfocal length, b is the aperture spacing, Δ is the disparity, and Z isthe depth. Values for F₁, F₂, and b may be selected to set the viewangle, magnification, and working distance of camera 100. For example,F₁ may range between 60-100 millimeters (mm)±10%, values for F₂ mayrange between 10-35 mm±10%, and values for b may range from 5 to 20mm±10%.

With the object 110 located in the front focal plane of objective 102, atradeoff between depth of focus, resolution and light level for roundapertures may be described by the following equations:Lateral Resolution (Rayleigh Criterion, Airy Disk Radius) at object110=1.22λ(F ₁ /D _(A))   (2)Diffraction−Limited Depth of Focus at object 110=±2λ(F ₁ /D _(A))²   (3)Light level is proportional to (D _(A) /F ₁)²   (4)where D_(A) is the diameter of the aperture and λ is the wavelength.Equations (2) and (3) apply to diffraction-limited imaging systems whileequation (4) applies to optical systems in general. Imaging system 100may be diffraction-limited.

In the event that imaging system 100 is telecentric at object 110, asdescribed above, its field of view (FOV) is limited by the diameterD_(L) of objective lens 102, focal length F₁, and F # of imaging system100 at the object 110, as follows:FOV≤D _(L) −F ₁ ×α−F ₁ /F #  (5),where α is the angle between the right and left viewing angles asmeasured in radians.

Since a large FOV typically requires a large objective, resulting in aheavy and bulky optical system, the FOV may be constrained to allowcamera 100 to have a size and weight that are suitable for a handhelddevice. To compensate for a smaller FOV, object 110 may be scanned tocapture many consecutive image pairs. Equation (1) may be applied toeach of the image pairs acquired using the stereo imaging system above,to calculate the depth attribute, or a 3D point cloud, for each imagepair. A registration algorithm may be used to add all the calculated 3Dpoint clouds together and form a large 3D point cloud representing thescanned region.

Any noise of a single 3D point cloud may be accumulated in theregistration process, resulting in a significant noise level for thelarge 3D point cloud. To limit the sensitivity to noise, camera 100 maybe designed such that the angle α between the left and right images (theimage pairs acquired in stereo), may be substantially small, such as˜6°. Alternatively, the angle α between the left and right images mayrange between 5.5° and 6.5°, or 5° and 7°, or 4° and 8° or 3° and 9°.Thus, the features in the left image and right image may be verysimilar, allowing a high degree of accuracy in discerning features alongthe lateral, x, and vertical, y axes. However, there may remain anon-negligible distortion along the depth, z axis.

Reference is now made to FIGS. 2A-2B which illustrate perpendicularviews of an optical imaging system 200 designed to achieve an expandedfield-of-view while maintaining careful control of imaging performance,in particular optical distortion, in accordance with another embodiment.

Distortion is an optical aberration which incorrectly maps points on theobject to the image. This incorrect mapping of points may have asubstantial effect on 3D point clouds. Following Eq. (1), therelationship dz between z (depth) and disparity (Δ) in a system with arelatively small angle between left and right images, such as ˜6° isdz˜10*Δ, where the disparity is calculated in the image plane. In such asystem, even very low distortion, such as tenths of a percent, may havea non-negligible effect on the large 3D point cloud. To avoid sucherrors, telecentric optical imaging system 200 may capture images withvery low distortion, such as <0.5%.

The distortion in an optical system is typically an inherent attributeof the optical design, and may vary with the depth; when in focus, anoptical system may acquire images with very low distortion, howeverimages acquired far from focus may suffer from high distortion. Toaddress this problem, the lenses 102, 214, 105, and 106 (where lens 106can be a sequential array of multiple lenses which allow, together, agreater degree of control over various imaging properties, such asdistortion) work together to minimize the distortion of the opticalsystem in-focus, and also to cause the distortion to change verygradually with defocus, resulting in relatively low distortion along theentire depth of focus. By reducing the distortion for each capturedimage, the cumulative error for the 3D point cloud resulting from theregistration may be reduced significantly.

The system of FIG. 2A and FIG. 2B is substantially similar to that ofFIGS. 1A-1B having two imaging systems for two optical paths eachcorresponding to a different viewing angle of object 110, with thenotable differences that: First, instead of one front lens 102 as in thesystem of FIG. 1A, there are three lenses (102, 214 and 105) in thesystem of FIGS. 2A-2B. Second, instead of one back lens 106 in eachoptical path of FIGS. 1A-1B (namely, lenses 106 a and 106 b), there is apair of back lenses for each optical path in FIGS. 1A-1B, namely, backlenses 106 aa and 106 ab in one optical path, and back lenses 106 ba and106 bb in the other optical path. These back lenses, together with thefront lenses, allow control of the distortion. Light reflected offobject 110 in two separate optical paths is collected by the set ofthree front lenses (102, 214 and 105). The light of each optical path iscollimated onto a different aperture of mask 104 and transmitted throughthe apertures via back lens pairs 106 aa+106 ab and 106 ba+106 bb ontosensors 108 a and 108 b, respectively.

The overall length of the system 200 of FIG. 2A may range from 100 to130 mm. In one embodiment, the overall length is approximately 120mm±10%. The maximum diameter of system 200 may range from 60 to 80 mm,and in one embodiment may be approximately 70 mm±10%. The averagedistortion may range from 0.4% to 0.6% and in one embodiment may beapproximately 0.5%. The FOV may range from 60×25 mm to 60×35 mm, and inone embodiment, may be approximately 60×30 mm±10%. The depth of focusmay range from ±15 mm to ±25 mm and in one embodiment, may beapproximately ±25 mm±10%. The depth of focus as described here takesinto account the geometric optics imaging behavior of system 200 as wellas the sampling of sensor 108, and may be different than thediffraction-limited depth of focus as defined above in equation (3). The3D resolution may range from 80 to 100 microns, and in one embodiment,may be 100 microns±10%.

Reference is now made to FIG. 3 which illustrates reflective propertiesof human skin, and which may be leveraged in order to select one or morespectral bandwidths that allow acquiring images detailing various skinfeatures. The spectral bandwidth can be controlled in the illuminationpath, the imaging path, or a combination. Skin typically absorbs lightin a wavelength dependent manner with the penetration depth generallyincreasing with the wavelength in the visible spectrum. Thus, redwavelengths are typically absorbed deeper into skin tissue than greenwavelengths, which are absorbed deeper than the blue range of thevisible spectrum, a property which may affect their respective surfaceand sub-surface reflection and scattering from the skin.

Thus, it may be appreciated that acquiring multiple images of the skinilluminated with different wavelengths having different specular anddiffuse and surface and sub-surface reflective properties may allowmeasurement of different skin characteristics both from the skin'ssurface and beneath. For example, the longer wavelength ranges thatpenetrate beneath the skin, such as the red and green ranges of thevisible spectrum, may be used to detect melanin and hemoglobin levels,and the surface reflective properties of blue light may be used todetect surface features of the skin. Multiple images may be acquiredunder illumination by different wavelengths using the system and methoddisclosed herein. These images may be combined with the 3D map createdabove to produce a multi-layer 3D reconstruction of the skin.

Optionally, a mask, such as mask 104 illustrated in FIGS. 1A, 2A, and2B, may be provided with multiple aperture pairs, each pair dedicated toa different wavelength range, to separately acquire multi-spectralimages of the skin, each indicating different skin characteristics.Details of mask 104 are described in greater detail below with respectto FIGS. 9A-9D. Each aperture of each aperture pair may correspond to adifferent one of the two viewing systems, allowing for multi-spectralstereo image acquisition.

Reference is now made to FIG. 4A which shows the spectral response ofvarious optical Bayer filters. In one embodiment, such filters may bedisposed with mask 104 of FIGS. 1A, 2A-2B, and 9A-9D when illuminating asample using a white light source, to detect surface and sub-surfacereflections under varying wavelengths. As indicated in FIG. 4A, Bayerfilters typically have a very large bandwidth, in the order of afull-width-at-half-maximum (FWHM) bandwidth of 100 nm. Thus, a Bayerfilter dedicated to transmitting blue light may transmit a substantialamount of green light that penetrated to the epidermis layer.Additionally, since typical Bayer filters have low transmission, and theblue light portion of the white LED spectrum may be relatively weak,collecting a sufficient amount of blue light for imaging requires a highintensity white light source.

Referring to FIG. 4B, the illumination bandwidths of several color LEDsis shown, each having a FWHM bandwidth in the range of 10 nm to 30 nm.It may be appreciated the bandwidth for each light source issubstantially narrower than the bandwidth allowed using the Bayerfilters of FIG. 4A. Thus, it may be appreciated that illuminating withmultiple color LEDs each having a narrow bandwidth, such as in the rangeof 10 nm, may allow detecting each wavelength range separately,precluding the collection of green light with blue light.

Typically, color sensors allocate one quarter of the pixels forcapturing blue light, and thus, sampling, or digital resolution for bluewavelengths is one half of the sensor resolution in each lateraldirection X and Y. Thus, in order to acquire a sufficient amount oflight to perform high resolution imaging, sensor 108 may be a monosensor that sequentially acquires images under illumination by multipledifferent monochromatic LEDs. Three monochromatic LEDs, corresponding tored, green, and blue wavelength ranges, may sequentially illuminatesample 110, allowing mono sensor 108 to separately detect eachindividually reflected wavelength range. Processor 112 may synchronizethe illumination cycle of the LEDs with the shutter of camera 100 or200, and may sort the detected images according to their illuminationwavelength range. The term “shutter” refers to either a mechanical(leaf, diaphragm, etc.) shutter, or to electronic shutter functionalityof the image sensor itself, which electronically “shades” sensor cells,as known in the art.

Alternatively, sensor 108 may be divided into different regions, eachregion allocated to detect a different one of the wavelength ranges.Thus, sensor 108 may have a region allocated for acquiring images of thetissue under blue, red, and green illumination, accordingly. Thisconfiguration allows simultaneously illuminating the tissue with allthree wavelength ranges, and capturing individual images for eachilluminating wavelength range.

Considerations for designing the system for either sequential orsimultaneous illumination and detection may include constraints on thesize of the detector: a small detector 108 may be more suited forsequential illumination-detection, whereas a larger detector 108 may bemore suited for simultaneous illumination-detection, having a greaternumber of pixels and/or detection area that can be divided and allocatedamong the different wavelength ranges while providing sufficientresolution.

Additionally, illumination-shutter synchronization, in either of thesequential or simultaneous illumination regimes, may be performed withillumination pulses which are substantially equal in length to shutteropening times, or with continuous illumination which spans temporallyover multiple shutter openings.

Reference is now made to FIGS. 5A-5C which show images of human skincaptured using varying illumination and sensors. FIG. 5A shows a skinsample illuminated using a blue LED having a full-width-at-half-maximum(FWHM) bandwidth of approximately 20 nm and detected using a monosensor. FIG. 5B shows the same skin sample illuminated using whiteillumination and detected using a color sensor. FIG. 5C shows an imagecreated from the blue pixels of the FIG. 5B. It may be appreciated thatsignificantly more details are evident in FIG. 5A than in any of FIGS.5B-5C. Thus, images may be acquired of the skin while illuminating witha blue LED. These images may be used for 3D reconstruction.

Reference is now made to FIG. 6 which shows visibility measured with aresolution target of 7 lines/millimeter (mm) as a function of distancefrom camera. Visibility is defined as (maximum intensity−minimumintensity)/(maximum intensity+minimum intensity). FIG. 6 showsvisibility of a standard resolution target captured using a mono sensor(curve 600) and a color sensor (curve 610) as measured at varyingdistances (in mm) from the camera. It may be appreciated that thevisibility using a mono sensor, indicated by curve 600, is consistentlyhigher and covers a broader distance range than with the color sensor,indicated by curve 610.

Polarization describes the electric field orientation with respect tothe meridional plane in an optical system, where S polarizationindicates a perpendicular orientation and P represents a parallelorientation. Illuminating with polarized light may provide additionalinformation about the skin. When the LEDs are oriented off-axis of theimaging path, their light may be S or P polarized using one or morepolarizers, independently of the imaging path.

Different combinations of parallel and/or crossed (orthogonal) polarizerpairs may be provided in the illumination and imaging paths: a linearpolarization component, typically referred to as the “polarizer” may beprovided with the LED to polarize the emitted light and illuminate thesurface with the polarized light, and a complementary linearpolarization component, typically referred to as the “analyzer”(parallel, crossed, or at another orientation to the polarizer) may beprovided at one or more of the apertures of mask 104, accordingly.Alternatively, the “analyzer” may be provided at the collecting lens 102or another suitable location in the optical path. Additionally, oralternatively, other polarization components, for example wave-platesand prisms may be provided in the illumination and/or imaging paths toachieved desired effects.

Optionally, the surface may be measured under four different types ofillumination and imaging conditions:

-   -   1) Non-polarized, or randomly polarized light, by omitting        polarization components from the optical path,    -   2) Parallel polarization in the illumination and imaging paths        by providing two polarizers that are parallel to each other: a        linear polarizer at one or more of LEDs 122 and a linear        polarizer along the imaging path, such as at mask 104,    -   3) Perpendicular polarization in the illumination and imaging        paths by providing two linear polarizers that are perpendicular        to each other, one linear polarizer at one or more of LEDs 122        and the other linear polarizer along the imaging path, such as        at mask 104, and    -   4) Arbitrary angle between the polarization of the illumination        path and the polarization of the imaging path, allowing a        combination of parallel and perpendicular polarization        conditions to be collected simultaneously. The ratio between the        parallel and perpendicular polarization conditions may be set by        the relative rotation angles of the linear polarizers in the        illumination and imaging paths. The ratio may also be made        adjustable to optimize the system for specific requirements or        conditions (skin tone, color, moisture, etc.)

Referring to FIGS. 7A-7C, images of human skin captured using a monosensor under blue illumination with no polarizers (FIG. 7A), mono sensorunder blue illumination with cross polarizers on the LEDs and imagingpath (FIG. 7B), and mono sensor under blue illumination and parallelpolarizers on the LEDs and imaging path (FIG. 7C), are shown. It may beappreciated that under illumination by blue light with parallelpolarizers, the 3D texture of the skin surface as indicated by the skinpores is discernible (FIG. 7B), whereas with the crossed polarizers(FIG. 7C) only the pigmentation is discernible. In FIG. 7A, without thepolarizers, both the skin surface and pigmentation are discernible.

When calculating 3D disparity maps from 2D images, the disparity iscalculated between features present in each of the image pairs acquiredin stereo. Pigmentation features may provide more accurate informationfor depth resolution, since textured skin features such as pores mayappear differently under varying lighting conditions. Since pigmentationis less dependent on lighting conditions, images captured using bluelight illumination with a mono sensor and no polarizer may be used for3D measurements.

Since melanin and hemoglobin are found under the surface of the skin,imaging using light free of surface reflection together with crossedpolarizers may provide more accurate data for measuring levels of thesecompounds.

Referring to FIGS. 8A-8D, images of a human skin sample captured usingparallel polarizers and crossed polarizers, respectively, are shown.FIG. 8A shows an image of the human skin sample captured using parallelpolarizers that collect light reflected from the skin surface, and FIG.8B shows an image of the same skin sample captured using crossedpolarizers that collect light that penetrated into the skin. FIGS. 8Cand 8D show images of the same skin sample captured under green and redillumination, respectively, with cross polarizers. It may be appreciatedthat use of the crossed polarizers enable the detection of lightscattered at much deeper layers of the skin.

Thus, acquiring images of the skin using polarized light of differentwavelengths may provide measurements of additional skin characteristics.Such images may be combined with the multi-layer 3D reconstructionproduced above as an additional layer.

Reference is now made to FIGS. 9A-9D which, taken together, illustrate asystem for multi-layer image acquisition for 3D reconstruction of anobject. The system 300 of FIG. 9A is substantially similar to the systemof FIGS. 1A-1B and 2A-2B having two viewing systems each comprising adifferent optical path corresponding to a different viewing angle of anobject, with the notable difference that an illumination source 122includes three individual light sources, such as red, green, blue lightemitting diodes (LEDs). An optical apparatus 126 is provided to operatewith the multiple illumination sources 122. Apparatus 126 includes mask104, one or more back lenses, such as lenses 106 a and 106 b, and one ormore detectors, such as detectors 108 a and 108 b. FIG. 9A shows aprofile view of mask 104 and FIGS. 9B-9D show exemplary front views ofmask 104 having multiple apertures for guiding light emitted by LEDs122.

The wavelength of the blue LED may range from 450 nm to 460 nm, or 455nm to 465 nm, or 460 nm to 470 nm, or 465 nm to 475 nm, or 470 nm to 480nm, or 475 nm to 485 nm, or 480 nm to 490 nm, or 485 nm to 495 nm, or450 nm to 465 nm, or 460 nm to 475 nm or 470 nm to 485 nm or 480 nm to495 nm, or 450 nm to 470 nm, or 460 nm to 480 nm, or 470 nm to 490 nm,or 475 nm to 495 nm.

The wavelength of the green LED may range from 495 nm to 505 nm, or 505nm to 515 nm, or 515 nm to 525 nm, or 525 nm to 535 nm, or 535 nm to 545nm, or 545 nm to 555 nm, or 555 nm to 565 nm, or 560 nm to 570 nm, or495 nm to 510 nm, or 510 nm to 525 nm, or 525 nm to 540 nm, or 540 to555 nm, or 555 nm to 570 nm, or 495 nm to 515 nm, or 515 nm to 535 nm,or 535 nm to 555 nm.

The wavelength of the red LED may range from 620 nm to 630 nm, or 630 nmto 640 nm, or 640 nm to 650 nm, or 650 nm to 660 nm, or 660 nm to 670nm, or 670 nm to 680 nm, or 680 nm to 690 nm, or 690 nm to 700 nm, or700 nm to 710 nm, or 710 nm to 720 nm, or 720 nm to 730 nm, or 730 nm to740 nm, or 740 nm to 750 nm, or 620 nm to 635 nm, or 635 nm to 650 nm,or 650 nm to 665 nm, or 665 nm to 680 nm, or 680 nm to 695 nm, or 695 nmto 710 nm, or 710 nm to 725 nm, or 725 nm to 740 nm, or 735 nm to 750nm, or 620 nm to 640 nm, or 640 nm to 660 nm, or 660 nm to 680 nm, or680 nm to 700 nm, or 700 nm to 720 nm, or 720 nm to 740 nm, or 730 nm to750 nm.

Mask 104 may be disposed with multiple different apertures pairs 114,116, and 118, optionally having varying shapes and/or sizes. Mask 104and or LEDs 122 may be positioned within the camera 300 such that eachaperture pair 114, 116, and 118 is dedicated to a different one of LEDs122 and/or light polarization to capture optical properties, includingbut not limited to specular and diffuse reflection, surface andsub-surface scattering, and absorption characteristics of sample 110.Optionally, each of aperture pairs 114, 116, and 118 may be opticallyaligned with a different one of LEDs 122. For example, each of aperturespairs 114, 116, and 118 may be provided with a different color filtersuch that, for example, aperture pair 114 may be dedicated totransmitting red wavelengths, aperture pair 116 may be dedicated totransmitting blue wavelengths, and aperture pair 118 may be dedicated totransmitting green wavelengths. This may allow simultaneous illuminationof sample 110 with the three LEDs 122 and capture of multiple images ofsample 110, an image for each LED 122 and for each of the two imagingsystems, at different dedicated sensors 108 or regions thereof.Alternatively, if such color filters are used, the illumination may bewhite-only, instead or red, blue, and green LEDs.

Optionally, any of the apertures of mask 104 may be disposed with apolarization component, such as an analyzer or retarder, complementing apolarization component configured with one of more of LEDs 122 allowingto detect one or more skin characteristics sensitive to polarization. Bydetecting an image of surface 110 separately via each aperture, multiplestereo images of surface 110 may be acquired indicating featuressensitive to different optical attributes, such as the illuminationwavelength, polarization, aperture shape, and the like.

Each side of mask 104, A and B, delineated by a dashed line forillustrative purposes, may correspond to a different one of the twoimaging systems, and thus each aperture of each of pairs 114, 116, and118 positioned on the respective sides of mask 104 may be dedicated totransmitting a light pulse along one of the two optical pathscorresponding to one of viewing angles θ₁, θ₂, allowing to determine thedepth characteristics indicating the 3D texture of object 110. Thus,light from each of LEDs 122 reflecting off object 110 at the twodifferent viewing angles θ₁, θ₂ may be transmitted via each of theviewing systems and the respective dedicated apertures 114 a, 116 a, and118 a disposed on side A and apertures 114 b, 116 b, and 118 b disposedon side B of mask 104 to detectors 108 a and 108 b or dedicated regionsof a monosensor 108, allowing for multi-spectral stereo imageacquisition. Processor 112 may combine the acquired images pairs toproduce a multi-layer 3D reconstruction of object 110. Processor 112 maydisplay the multi-layer 3D reconstruction on a user display 124.

Referring to FIG. 9B, an implementation for a multi-aperture mask 104 isshown, having aperture pairs 114, 116, and 118 comprising apertures 114a, 114 b, 116 a, 116 b and 118 a, 118 b, disposed on sides A and B,respectively, allowing for multi-layer image acquisition. Each of theapertures and/or the aperture pairs 114, 116, and 118 may be of adifferent size and/or geometric shape. The aperture diameters andgeometric shapes may vary in accordance with the required resolution,light intensity, and depth of focus. The apertures of each pair may bepositioned symmetrically opposite to each other about vertical andhorizontal diameters of mask 104, and about the viewing axis of system300. Each of the two optical paths may be directed to a differentaperture of each pair, allowing stereo image acquisition in accordancewith the attributes designated for each aperture pair.

Referring to FIG. 9C, mask 104 is shown as substantially similar to mask104 of FIG. 9B, with the noted difference that the apertures of eachpair have a matching shape resulting in a symmetry about the diameter ofmask 104. In this example, apertures 114 a and 114 b are longitudinallydisposed rectangular slits, apertures 116 a and 116 b are squares, andapertures 118 a and 118 b are round, however this is not meant to belimiting and other shapes, or arrangements may be used.

For stereo vision two identical or similar apertures may be provided,separated to view surface 110 at angles that differ at least byapproximately 6°. To provide for a sufficiently large area on the tworegions of sensor 108, or sensors 108 a and 108 b for 3D imaging, atleast the two central positioned apertures 116 a, and 116 b of the maskfor 3D may be used. The other apertures (114 a, 114 b, 118 a, and 118 b)may be used separately with each aperture optionally having differentsize, and shape. The size and shape of the apertures determines theresolution, depth of focus and light level according to Eq. 2-4. Roundapertures give the same resolution in both directions, the larger theaperture the higher the resolution and light level but the lower thedepth of focus. Other shapes may be used to capture features thatrequire different resolutions in different directions. For example, todetect 2D and/or 3D features on the skin, such as wrinkles, stretchmarks, etc., that require high spatial resolution in one direction andlower spatial resolution in the other direction, an aperture that isnarrow in one direction and wide on the other direction may be used,such as may have an elliptical or rectangle shape. Detecting the lightfrom multiple different directions may also enable a betterreconstruction of the 3D map of object 110.

Light rays passing through the apertures positioned on one side of mask104 (i.e. the side marked ‘A’) shown as apertures 114 a, 116 a, and 118a, may be directed towards sensor 108 a positioned on the left side ofthe viewing axis, while light rays passing through apertures 114 b, 116b, and 118 b positioned on the side of mask 104 marked ‘B’ may bedirected to sensor 108 b, positioned on the right side of the viewingaxis. Light passing through the upper and lower apertures 114 a, 114 band 118 a, 118 b may be focused onto sensors 108 a and 108 b atdifferent angles than light passing the centrally positioned apertures116 a and 116 b. The light passing through mask 104 may be detected arededicated regions of a single sensor 108 or at one or more smaller,dedicated sensors, described in greater detail below.

Reference is now made to FIG. 9D which illustrates an exemplaryimplementation of mask 104 disposed with one or more polarizationcomponents. Surface 110 may be illuminated via one or more light sources122 disposed with one or more polarization components, and one or moreapertures dedicated to the LED with the polarization component may bedisposed with a complementary polarization component. These pairedpolarization components may be parallel or crossed (orthogonal) withrespect to each other, with ‘S’ and ‘P’ as defined above.

Thus, for aperture pair 114 dedicated to red light, one aperture (114 a)may be have a parallel polarization component (P) and the other aperture(114 b) may have a perpendicular polarization component (S). Foraperture pair 116 dedicated to blue light, both apertures (116 a, 116 b)may have a parallel polarization component (P). For aperture pair 118dedicated to green light, one aperture (118 a) may be have a parallelpolarization component (P) and the other aperture (118 b) may have aperpendicular polarization component (S). However, this implementationis meant to be illustrative only, and is not meant to be limiting.

Alternatively, other polarization components may be used, such asquarter-wave and half-wave plates for the differentcolor/aperture-size/shape apertures of mask 104.

Optionally, the polarization for each aperture may be as indicated inFIG. 9D: apertures 114 a, 116 a, 118 a, and 114 b, 116 b, and 118 b mayhave polarizers that are parallel or crossed (orthogonal) compared tothe polarizers on the LED's (120 for example) or alternatively not havepolarizers at all.

Since blue light does not penetrate very deeply into the skin, bluelight may be used for collecting light from the surface of object 110.Blue light is primarily reflected from the skin's surface as bothspecular and diffuse reflectance. Since the imaging system disclosedherein collects a small angular range due to the high F/#, and thediffuse reflection is spread into a relatively large angular range, andthus a relatively small amount of light from each LED is transmittedthrough the aperture, primarily comprising specular reflection.Accordingly, blue light source 122 may be provided with a polarizer 120and middle apertures 116 a and 116 b dedicated to transmitting the bluelight may be provided with a parallel polarizer (not shown), such as apolarized coating, allowing the surface reflection of the polarized bluelight to be transmitted via dedicated aperture pair 116 to detector 108.These images acquired using blue light may be used by processor 112 tomeasure the depth characteristics of skin surface 110.

Conversely, light that undergoes diffuse reflection or scattering maychange its polarization state. Thus, one or more perpendicularpolarizers (not shown) may be provided between the illumination source122 and the imaging path to emphasize diffuse skin reflection, as wellas sub-surface scattering. Since the red and green wavelengths penetratedeeper into the skin, apertures 118 a and 118 b may be dedicated totransmitting red light may be disposed with a polarizer that isperpendicular with respect to a polarizer disposed with the red LED 122.Similarly, apertures 114 a and 114 b may be dedicated to transmittinggreen light and may be disposed with a polarizer that is perpendicularwith respect to a polarizer disposed with the green LED 122.

For example, polarization may be used to capture images for measuringhemoglobin and melanin concentrations of the skin 110. Thus, skinsurface 110 may be illuminated using the red and green LEDs 122, and theresulting diffused and/or scattered reflection from deeper layers ofskin surface 110 may be transmitted via apertures 118 b and 114 b,respectively, and imaged at sensor 108 b, or at regions of sensor 108.These images may be analyzed by processor 112 to measure the hemoglobinand melanin levels, and superimposed by processor 112 as ‘layers’ on the3D model reconstructed above.

The color characterization of skin surface 110 may be measured usingmulti-spectral acquisition, by sequentially or simultaneouslyilluminating the skin surface 110 using each of the red, blue, and greenLEDs 122. Each reflected wavelength may be separately transmitted viaone aperture of its respective dedicated aperture pair, 114, 116, and118, and detected by mono sensor 108. Optionally, three images: oneimage per wavelength, may be superimposed to create a two-dimensional(2D) color image of skin sample 110 which may be combined with the 3Dmodel and optionally with the hemoglobin and/or melanin reconstruction,to create a multi-layer 3D reconstruction of skin sample 110. Forexample, images captured at sensor 108 a via apertures 114 a, 116 a, and118 a may be used for the 2D color image, and which may be overlaid onthe 3D reconstruction as a color layer. Images captured at sensor 108 bvia apertures 114 b, 116 b, and 118 b may be used for subsurfaceanalyzing of the skin.

Optionally, white light may be synthesized from suitable ratios ofillumination by the red, green, and blue LEDs. When acquiring themulti-layer images, LEDs 122 may be synchronized with one or moreshutters of camera 100 or 200. The shutters may operate in short pulsescorresponding to the red green and blue illumination sequence.Optionally, for each pulse, both sides 108 a and 108 b of sensor 108 maysimultaneously capture two stereo images corresponding to the twooptical paths via any of the aperture pairs of mask 104.

Imaging thus in stereo using varying wavelengths and/or polarization mayallow detecting both depth and spectral characterization of object 110.

Referring to FIGS. 9E and 9F, another configuration for opticalapparatus 126 is shown (in perspective and side views, respectively),having 6 sensors, 108 a-108 f, each dedicated to a different one of thesix apertures 114 a, 114 b, 116 a, 116 b, 118 a, 118 b of mask 104. Theapertures are shown having a uniform size and shape for illustrativepurposes only, and may optionally have differing sizes and shapes asdescribed above. Optionally, each aperture pair may be provided with adifferent color filter and/or polarizer corresponding to LEDs 122, asdescribed above. Each aperture 114 a, 114 b, 116 a, 116 b, 118 a, 118 bmay have a dedicated back lens, 106(R)a, 106(B)a, 106(G)a, 106(R)b,106(B)b, and 106(G)b, respectively, shown as two sets of three backlenses corresponding to stereo three-color detection, where ‘R’ isunderstood to represent red, ‘B’ is understood to represent blue, and‘G’ is understood to represent green. Each lens may be aligned toseparately focus light from each of the six apertures and produce animage on different dedicated sensors 108 a-108 f, or different regionsof sensor 108.

Thus, in one embodiment, apertures 114 a and 114 b may optionally beprovided with a red color filter and optionally a polarizing component(not shown) corresponding to a polarizing component disposed with redLED 122. Sensors 108(R)a and 108(R)b may be aligned to sense red lightemitted via apertures 114 a and 114 b and focused via lenses 106(R)a and106(R)b and capture red stereo images of surface 110. Apertures 116 aand 116 b may be provided with a blue color filter and optionally apolarizing component (not shown) corresponding to a polarizing componentdisposed with blue LED 122. Sensors 108(B)a and 108(B)b may be alignedto sense light emitted via apertures 116 a and 116 b, and focused vialenses 106(B)a and 106(B)b and dedicated to capturing blue stereo imagesof surface 110. Apertures 118 a and 118 b may be provided with a greencolor filter and optionally a polarizing component (not shown)corresponding to a polarizing component disposed with green LED 122.Sensors 108(G)a and 108(G)b may be aligned to sense light emitted viaapertures 118 a and 118 b and focused via lenses 106(G)a and 106(G)b,and dedicated to capturing green stereo images of surface 110. Themultiple images captured thus, optionally simultaneously, may be used toconstruct a multi-layer, high resolution, 3D image of surface 110. Itmay be appreciated that the order of the colors is arbitrary and otherarrangements may be used.

Reference is now made to FIG. 9G, which shows another configuration foroptical apparatus 126. The system of FIG. 9G is substantially similar tothat of FIGS. 9E-9F with the noted exception that two lenses and twosensors are provided, a shared lens and sensor dedicated to each of thetwo imaging systems or stereoscopic views, corresponding to each side A,B, of mask 104. Light passing through the apertures 114 a, 116 a, and118 a on side A of the mask 104 and the dedicated lens 106 a will reachsensor 108 a at the same position but different angles. Similarly, lightpassing through the apertures 114 b, 116 b, and 118 b on side B of mask104 and the dedicated lens 106 b will reach sensor 108 b at the sameposition but different angles. Detecting the light from multipledifferent directions may enhance the reconstruction of the 3D map ofobject 110.

In this configuration, LEDs 122 may illuminate the surface sequentially,allowing sensors 108 a and 108 b to simultaneously capture a pair ofimages for each illuminating LED 122, precluding the need to providecolor filters with mask 104.

Reference is now made to FIGS. 10A-10D, each showing three RGB cyclesfor multi-layer image acquisition using the mask of FIG. 9C. FIG. 10Agenerally illustrates the cycles; FIG. 10B illustrates the cycleswithout polarization; FIG. 10C illustrates the cycles at sensor 108 awith certain polarization; and FIG. 10D illustrates the cycles at sensor108 b with certain polarization.

Image acquisition by sensors 108 a and 108 b may be synchronized withillumination source 112. For example, processor 112 may synchronizeillumination source 122 with a shutter (not shown) of camera 300. FIG.10A shows three illumination cycles of surface 110, each cyclecomprising sequential blue, red, and green illumination pulses by thecolored LEDs of illumination source 122 for a total of 9 pulses. FIGS.10C and 10D each show three image acquisition cycles by sensors 108 aand 108 b, respectively, synchronized with the illumination cycles ofFIG. 10A, resulting in the acquisition of an image per pulse for eachsensor. Apertures of mask 104 corresponding to images acquired usingparallel-polarized light are indicated as ‘P’, and apertures of mask 104images acquired using cross-polarized light are indicated as ‘S’. Theparallel-polarized illumination allows measuring the skin surface colorand/or depth corresponding to the specular reflection, and the polarizedillumination allows measuring light that has penetrated into deeperparts of the skin.

As seen in FIGS. 10A-10D, for each illuminating pulse by any of the LEDsof illumination source 122, two images are simultaneously captured viathe two optical paths described above: one image captured at sensor 108Aand the other image captured at sensor 108B. Each pair of acquiredimages may be used to measure different characteristics of object 110,and which may be processed by processor 112 to reconstruct amulti-layered 3D model of object 110. Thus, in FIGS. 10C-10D, for eachblue illumination pulse, two parallel-polarized images are captured, oneimage at each of sensors 108 a and 108 b. The disparity between theseimages may be used to calculate the depth characteristic of object 110.For each of the red and green illumination pulses, the image captured atsensor 108 a is acquired using parallel-polarized light, and the imagecaptured at sensor 108 b is acquired using cross-polarized lightcompared to the polarizer on the LED's. These images may be used tomeasure features or characteristics beneath the skin's surface.

Processor 112 may receive and register the images acquired above into anRGB image. The first step after calculating the disparity and the depthmap obtained by applying Eq. 1 to the images captured by apertures 116 aand 116 b, may be to shift images captured by different apertures usingthe depth map results by the following equation:

$\begin{matrix}{{{shift}\left( {i,j} \right)} = \frac{\left( {{z\left( {i,j} \right)} - F_{1}} \right)F_{2}b}{F_{1}^{2}}} & (6)\end{matrix}$Where (i, j) are the index of the pixel on the image, z is the depthcalculated from the disparity map and F₁ and F₂ are the focal lengths asdescribed by Eq. 1.

Optionally, the relative position of each of the acquired images may beobtained, such as by detecting their respective offset and rotation.This may be implemented by searching the relative position for eachimage that maximizes the correlation between each image pairsimultaneously acquired for each illumination pulse.

Optionally, if the discrepancies between the images is large, a directcorrelation may be difficult to calculate. Referring to FIGS. 11A-11C,images of the same skin surface captured by sensor 108 under blue,green, and red illumination, respectively, are shown. It may beappreciated that the discrepancies between the three images may pose achallenge to calculating a direct correlation therebetween. To overcomethis, processor 112 may apply one or more image processing techniques,such as but not limited to, edge enhancement and equalization. Forexample, processor 112 may perform the following steps:

1. Calculate variance map for each color,

2. Scale all variance maps into the same dynamic range, and

3. Find relative positions that maximize correlation.

-   The resulting processed images may be used to reconstruct the 3D    model. Referring to FIG. 12, an exemplary multi-layer 3D    reconstruction is shown, using the system and method described    above.

Reference is now made to FIGS. 13A-B which taken together, show aflowchart of a method for creating a multi-layer 3D reconstruction of anobject, in accordance with an embodiment, and which may be implementedby one or more processors, such as processor 112. An illumination cyclecomprising controlling the illumination of multiple different coloredLEDs may be controlled. For example, the illumination cycle may compriseany combination of a blue illumination pulse by a blue LED, a greenillumination pulse by a green LED, and a red illumination pulse by a redLED (Step 400). A camera shutter may be synchronized with eachillumination pulse of the illumination cycle (Step 402). A pair ofimages of an object may be acquired during each illumination pulse (Step404). For example, the images may be acquired via a single mono sensordivided into two portions and configured with a stereo imaging system.Optionally, one or more images acquired during the illumination cyclemay be acquired from polarized light. Optionally, the polarized lightcorresponds to the blue illumination pulse, or any of the green and redillumination pulses.

A depth characteristic of the object may be calculated from one of thepairs of images acquired from a specular reflection of one of theillumination pulses (Step 406). The pixels of the captured images may beshifted as a function of the depth characteristic, and the focal lengthsF₁ and F₂ of Eq. 1. (Step 408). For example, the specular reflection maycorrespond to the blue illumination pulse. A sub-surface quality of theobject may be measured using any images acquired from a diffusereflection of one of the illumination pulses (Step 410). For example,the diffuse reflection may correspond to any of the green and redillumination pulses and the subsurface quality may comprise any ofmelanin and hemoglobin. A color characteristic of the object may bedetermined from any of the acquired images (Step 412). For example, thecolor characteristic may be determined from the image pair acquiredduring the blue illumination pulse, or from one or more images acquiredduring a single illumination cycle. For example, the colorcharacteristic may be determined from one image from each pair of imagesacquired during each of the blue, red, and green illumination pulses.

The pairs of images acquired over multiple illumination cycles may beregistered (Step 414). Optionally, any of an edge enhancement technique,an equalization technique, and an image correlation techniquecomprising: a) calculating a variance map for each color, b) scaling thevariance maps to the same dynamic range, and c) finding the relativepositions of detected features to maximize correlation may be applied tothe registered image pairs. The registered pairs of images may becombined to create a multi-layer 3D reconstruction of the object (Step416). The 3D reconstruction may be displayed on a user interface (Step418).

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a non-transitory, tangibledevice that can retain and store instructions for use by an instructionexecution device. The computer readable storage medium may be, forexample, but is not limited to, an electronic storage device, a magneticstorage device, an optical storage device, an electromagnetic storagedevice, a semiconductor storage device, or any suitable combination ofthe foregoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, or any suitable combination of the foregoing. A computerreadable storage medium, as used herein, is not to be construed as beingtransitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention may be described herein with referenceto flowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A system, comprising: A. a camera configured tooperate under telecentric conditions, said camera comprises: two imagingsystems each comprising a different optical path corresponding to adifferent viewing angle of an object; multiple illumination sources; amask comprising multiple pairs of apertures, wherein each aperture ofeach aperture pair corresponds to a different one of the imagingsystems, wherein each aperture of each of said pairs of apertures isconfigured to transmit a light pulse from a different optical path to atleast one imaging sensor, wherein said multiple pairs of apertures arepositioned symmetrically opposite about the viewing axis of said camera;and at least one detector configured to acquire multiple image pairs ofthe object from the two imaging systems via the multiple pairs ofapertures; and B. a processor configured to produce, from the multipleacquired image pairs, a multi-layer three dimensional reconstruction ofthe object wherein said viewing angle is between 3 and 9 degrees.
 2. Thesystem of claim 1, wherein each pair of apertures is dedicated to adifferent one of the multiple illumination sources.
 3. The systemaccording to claim 1, where said multiple illumination sources comprisea red LED, a green LED, and a blue LED.
 4. The system of claim 3,wherein each of the multiple different LEDs has a full-width-at-half(FWHM) bandwidth of 10 nm±10%.
 5. The system according to claim 1,wherein the at least one detector comprises a mono sensor having adedicated region for each aperture, and configured to simultaneouslycapture multiple images comprising an image for each of the multipleillumination sources and for each of the imaging systems.
 6. The systemaccording to claim 1, wherein the at least one detector comprises twoimaging sensors that are each dedicated to one of the imaging systemsand are configured to simultaneously detect an image pair for each ofthe illumination sources.
 7. The system according to claim 1, whereinthe at least one detector comprises an imaging sensor having multiplesensor regions, each sensor region dedicated to a different one of themultiple apertures, and wherein the multiple sensor regions areconfigured to simultaneously capture multiple images comprising an imagefor each of the multiple illumination sources and for each of theimaging systems.
 8. The system according to claim 1, wherein at leastone of the multiple different illumination sources is disposed with apolarization component, and wherein at least one of the aperturesdedicated to the illumination source disposed with the polarizationcomponent, is disposed with a complementary polarization component. 9.The system of claim 8, wherein the at least one of the aperturesdisposed with the polarization component is dedicated to a blue lightsource.
 10. The system according to claim 1, wherein each pair ofapertures is disposed with a color filter corresponding to its dedicatedillumination source.
 11. The system according to claim 1, furthercomprising at least one back lens configured to focus light transmittedvia the multiple pairs of apertures of the mask to the at least onedetector.
 12. The system according to claim 1, wherein at least oneimage pair of the acquired image pairs corresponds to a specularreflection off the object, and wherein the processor is furtherconfigured to use the at least one image pair to measure a depthcharacteristic of the object.
 13. The system of claim 12, wherein theprocessor is further configured to apply a shift to the pixels of theacquired image as a function of the depth characteristic.
 14. The systemaccording to claim 1, wherein at least one of the acquired imagescorresponds to a diffuse reflection off the object, and wherein theprocessor is further configured to use the at least one acquired imageto measure a hemoglobin level of the object.
 15. The system according toclaim 1, wherein at least one of the acquired images corresponds to adiffuse reflection off the object, and wherein the processor is furtherconfigured to use the at least acquired one image to measure a melaninlevel of the object.
 16. The system according to claim 1, wherein theprocessor is configured to use at least one of the acquired images tomeasure a color characteristic of the object.
 17. The system accordingto claim 1, wherein the processor is configured to synchronize theillumination source with a shutter of the camera.
 18. The systemaccording to claim 1, further comprising a user interface configured todisplay the multi-layer three dimensional reconstruction of the object.19. The system according to claim 1, wherein said processor isconfigured to produce a three-dimensional (3D) point cloud representingthe object, based, at least in part, on said multiple image pairs of theobject.
 20. The system according to claim 19, wherein said multipleimage pairs of the object are a series of consecutive image pairs eachrepresenting a region of said object, wherein said processor isconfigured to produce a 3D point from each image pair of the series ofconsecutive image pairs, and wherein said processor is furtherconfigured to add all the 3D point clouds together into a single 3Dpoint cloud representing said object.